Data Models for Dataset Drift Controls in Machine Learning With Images

Camera images are ubiquitous in machine learning research. They also play a central role in the delivery of important services spanning medicine and environmental surveying. However, the application of machine learning models in these domains has been limited because of robustness concerns. A primary failure mode are performance drops due to differences between the training and deployment data. While there are methods to prospectively validate the robustness of machine learning models to such dataset drifts, existing approaches do not account for explicit models of the primary object of interest: the data. This makes it difficult to create physically faithful drift test cases or to provide specifications of data models that should be avoided when deploying a machine learning model. In this study, we demonstrate how these shortcomings can be overcome by pairing machine learning robustness validation with physical optics. We examine the role raw sensor data and differentiable data models can play in controlling performance risks related to image dataset drift. The findings are distilled into three applications. First, drift synthesis enables the controlled generation of physically faithful drift test cases. The experiments presented here show that the average decrease in model performance is ten to four times less severe than under post-hoc augmentation testing. Second, the gradient connection between task and data models allows for drift forensics that can be used to specify performance-sensitive data models which should be avoided during deployment of a machine learning model. Third, drift adjustment opens up the possibility for processing adjustments in the face of drift. This can lead to speed up and stabilization of classifier training at a margin of up to 20% in validation accuracy. A guide to access the open code and datasets is available at https://github.com/aiaudit-org/raw2logit.Comment: LO and MA contributed equall

Data models for dataset drift controls in machine learning with optical images

Camera images are ubiquitous in machine learning research. They also play a central role in the delivery of important public services spanning medicine or environmental surveying. However, the application of machine learning models in these domains has been limited because of robustness concerns. A primary failure mode are performance drops due to differences between the training and deployment data. While there are methods to prospectively validate the robustness of machine learning models to such dataset drifts, existing approaches do not account for explicit models of machine learning’s primary object of interest: the data. This limits our ability to study and understand the relationship between data generation and downstream machine learning model performance in a physically accurate manner. In this study, we demonstrate how to overcome this limitation by pairing traditional machine learning with physical optics to obtain explicit and differentiable data models. We demonstrate how such data models can be constructed for image data and used to control downstream machine learning model performance related to dataset drift. The findings are distilled into three applications. First, drift synthesis enables the controlled generation of physically faithful drift test cases to power model selection and targeted generalization. Second, the gradient connection between machine learning task model and data model allows advanced, precise tolerancing of task model sensitivity to changes in the data generation. These drift forensics can be used to precisely specify the acceptable data environments in which a task model may be run. Third, drift optimization opens up the possibility to create drifts that can help the task model learn better faster, effectively optimizing the data generating process itself to support the downstream machine vision task. This is an interesting upgrade to existing imaging pipelines which traditionally have been optimized to be consumed by human users but not machine learning models. The data models require access to raw sensor images as commonly processed at scale in industry domains such as microscopy, biomedicine, autonomous vehicles or remote sensing. Alongside the data model code we release two datasets to the public that we collected as part of this work. In total, the two datasets, Raw-Microscopy and Raw-Drone, comprise 1,488 scientifically calibrated reference raw sensor measurements, 8,928 raw intensity variations as well as 17,856 images processed through twelve data models with different configurations. A guide to access the open code and datasets is available at https://github.com/aiaudit-org/raw2logit

Data models for dataset drift controls in machine learning with optical images

Camera images are ubiquitous in machine learning research. They also play a central role in the delivery of important services spanning medicine and environmental surveying. However, the application of machine learning models in these domains has been limited because of robustness concerns. A primary failure mode are performance drops due to differences between the training and deployment data. While there are methods to prospectively validate the robustness of machine learning models to such dataset drifts, existing approaches do not account for explicit models of the primary object of interest: the data. This limits our ability to study and understand the relationship between data generation and downstream machine learning model performance in a physically accurate manner. In this study, we demonstrate how to overcome this limitation by pairing traditional machine learning with physical optics to obtain explicit and differentiable data models. We demonstrate how such data models can be constructed for image data and used to control downstream machine learning model performance related to dataset drift. The findings are distilled into three applications. First, drift synthesis enables the controlled generation of physically faithful drift test cases to power model selection and targeted generalization. Second, the gradient connection between machine learning task model and data model allows advanced, precise tolerancing of task model sensitivity to changes in the data generation. These drift forensics can be used to precisely specify the acceptable data environments in which a task model may be run. Third, drift optimization opens up the possibility to create drifts that can help the task model learn better faster, effectively optimizing the data generating process itself. A guide to access the open code and datasets is available at https://github.com/aiaudit-org/raw2logit

Progressive disruption of hematopoietic architecture from clonal hematopoiesis to MDS

Summary: Clonal hematopoiesis of indeterminate potential (CHIP) describes the age-related acquisition of somatic mutations in hematopoietic stem/progenitor cells (HSPC) leading to clonal blood cell expansion. Although CHIP mutations drive myeloid malignancies like myelodysplastic syndromes (MDS) it is unknown if clonal expansion is attributable to changes in cell type kinetics, or involves reorganization of the hematopoietic hierarchy. Using computational modeling we analyzed differentiation and proliferation kinetics of cultured hematopoietic stem cells (HSC) from 8 healthy individuals, 7 CHIP, and 10 MDS patients. While the standard hematopoietic hierarchy explained HSPC kinetics in healthy samples, 57% of CHIP and 70% of MDS samples were best described with alternative hierarchies. Deregulated kinetics were found at various HSPC compartments with high inter-individual heterogeneity in CHIP and MDS, while altered HSC rates were most relevant in MDS. Quantifying kinetic heterogeneity in detail, we show that reorganization of the HSPC compartment is already detectable in the premalignant CHIP state

Life satisfaction two-years after stroke onset: the effects of gender, occupational status, memory function and quality of life among stroke patients (Newsqol) and their family caregivers (Whoqol-bref) in Luxembourg

Life satisfaction (LS) of cerebrovascular disease survivors and their family caregivers may relate to socioeconomic factors, impaired functions, health-related quality of life (QoL), but their respective influences remain unclear. This study assessed, two years post-stroke onset, the effects of these factors on patients' LS and family caregivers' LS in Luxembourg. METHODS: All stroke patients admitted to all hospitals in Luxembourg were identified by the 'Inspection Général de la Sécurité Sociale' using the only national system database for care expenditure reimbursement. Their diagnosis was confirmed by medical investigator. The sample included ninety four patients living at home having given consent (mean age 65.5 years) and sixty two main caregivers (mean age 59.3 years). Questionnaires were completed during face-to-face interviews. LS was assessed via European single question (range 1-10), survivors' QoL via Newsqol (11 dimensions), and caregivers' QoL via Whoqol-bref (4 domains) (range 0-100). Data were analysed using multiple regression models. RESULTS: Two years after stroke onset, 44.7% of patients suffered from impaired sensory function, 35.1% from impaired motor function, and 31.9% from impaired memory function. Mean patient' LS was 7.1/10 (SD 1.9). It was higher in women (+12.4) and lower among unemployed socioeconomically active patients (-13.1, vs. retired people). Adjusted for sex, occupation, impaired motor and memory functions, LS positively correlated with scores of Newsqol feelings, sleep, emotion, cognition and pain dimensions (slopes 0.20 to 0.31), but did not correlate with those of caregivers' Whoqol-bref domains. Family caregiver' LS was 7.2 (SD 1.7). It was lower in those with patients suffering from impaired memory function (-12.8) as well as from feelings and emotion issues (slopes 0.22). It was associated with all caregivers' Whoqol-bref domains (physical health, psychological health, environment, and social relationships) (slopes 0.53 to 0.68). CONCLUSIONS: Two-year post-cerebrovascular disease patient' LS was associated with gender, occupation, and impaired memory function. It correlated with feelings, sleep, emotion, cognition, and pain issues. Family caregivers of patients with impaired memory function had lower LS. Family caregiver' LS correlated with dimensions of patients' feelings (less independent, yourself, life changed, depressed, useless, less control because of stroke) and emotion (get more emotional, fear of another stroke or to become dependent on others), and with their own QoL. LS, Newsqol, and Whoqol appeared to be appropriate tools. Our findings may be useful for policy makers in relation to family and medical-social issues of stroke home-based rehabilitation

New national and regional bryophyte records, 48

International audience[No abstract available